A digital photofinishing system includes a film scanner, an image data manager (IDM), and a printer to make photographic prints. The system is designed to scan photographic color negative and reversal camera films, apply a set of image processing steps to those images in the IDM, and print them to photographic paper.
Specifically, the system is designed for high volume photofinishing which is commonly used to produce prints for non-professional consumer customers. These systems are commonly used in wholesale and minilab applications, where high productivity, measured by the number of images processed in a specific period of time, is very important.
Random signal variations in a digital image are known as "noise." Excessive levels of noise are objectionable to customers. One major source of noise is the granularity of the film that is scanned. Film granularity increases with film speed, i.e., from ISO 100 to 800 speed, and increases with smaller format, e.g., from 35 mm to the Advanced Photo System (APS). The granularity of films generally increases with the degree of under-exposure.
A specific problem arises with certain image processing steps that enhance the appearance of noise in the images. Specifically, sharpening can enhance the noise to an objectionable level. This is of particular concern for high speed films, like ISO 800 speed, for smaller format films, like APS, and for all under-exposed film images.
At the sharpening step, an unsharp mask is applied to the image. The general equation for the unsharp mask is D.sub.sharp =D.sub.orig +K*(D.sub.orig -D.sub.blurred) where D.sub.sharp is the sharpened image, D.sub.orig is the original image, D.sub.blurred is a blurred version of the image, and K is a scalar constant.
FIGS. 1-3 conceptually demonstrate the trade-offs between random variations on density, referred to as noise, and sharpening levels. Plots are shown of 30 representative pixels of a flat field area of a digital image. The y-axis represents the deviation of each pixel code value from a density code value average for the flat field. The units of the code value deviations are arbitrary, and the density code value average is simply specified as zero.
The dash/dot lines, called the "objection limit," represent a threshold beyond which the density variations become objectionable to someone viewing the images. The objection limit in FIGS. 1-3 is arbitrary because it varies considerably depending on many factors, including the inherent modulation transfer functions for the printer and the photographic paper. Also, the size of the output images and the viewing conditions, such as the illuminant, are factors that affect the objection limit.
The objection limit can be a function of absolute density variations, a standard deviation of density variations, or some other statistical measurement.
FIG. 1 is the plot of 30 representative pixels of a flat field area of a digital image with no sharpening applied. All the density code value variations are well within the objection limit. The density code value variations will be mostly dependent on the inherent granularity of the camera film.
FIG. 2 is a plot of the same 30 pixels with a certain arbitrary level of sharpening applied. The exact level of sharpening depends on the blurring filter and the constant, K, from the unsharp masking equation. All the density code value variations have increased, due to the sharpening process, to a level where some of the variations approach the objection limit.
The sharpening demonstrated in FIG. 2 represents an acceptable level, based on the level of noise, to someone viewing the final image.
FIG. 3 is a plot of the same 30 pixels with a higher arbitrary level of sharpening applied. This situation represents where the constant, K, from the unsharp mask equation is higher than for the situation depicted in FIG. 2. All the density code value variations have increased, due to the sharpening process, to a level where some of the variations are beyond the objection limit.
The sharpening demonstrated in FIG. 3 represents an unacceptable level, based on the excessive level of noise, to someone viewing the final image.
Adjusting the value, K, in the unsharp mask equation is the most common way to control the amount of sharpening in an image. In the following three examples, the sharpening level is varied by adjusting the value, K, pixel-by-pixel based on a calculated estimate of the noise in the immediate region around each pixel. The noise calculation is performed on a neighborhood of pixels surrounding the pixel to be modified.
Mahmoodi and Nelson (U.S. Pat. No. 4,571,635) describe a method for adjusting the value, K, based on the calculated standard deviation of pixel values in the neighborhood immediately surrounding the pixel to be modified. The value, K, varies pixel-by-pixel.
Kwon and Liang (U.S. Pat. No. 5,081,692) describe a method for varying the sharpening by calculating the value, K, with a "centered weighted variance process." The value, K, varies pixel-by-pixel. This process estimates the noise in the neighborhood surrounding the pixel to be modified by weighting each of the neighborhood pixels depending on their location within the neighborhood.
Ishihara, Yamashita, and Fukushima (U.S. Pat. No. 5,390,264) describe a method where the value, K, is a function of the difference in values between the pixel to be modified and selected pixels in the surrounding neighborhood.
While varying the sharpening level pixel-by-pixel may be desirable, the methods which use this approach are demanding relative to the computing time required. This is counter to the need in consumer photofinishing applications for high productivity, which is measured by the number of images processed in a specific period of time.
Also, while methods which vary the sharpening level pixel-by-pixel are frequently appropriate for images for professional customers, they provide image quality benefits which are neither desired nor appreciated by non-professional consumers.
In addition, there are other methods for adjusting the sharpening level.
Shimazaki U.S. Pat. No. 5,051,842 describes an apparatus which generates unsharp signals from images, derives two parameters based on either the image signal level or the unsharp signal level from a pre-determined lookup table, multiplies one parameter with the image signal, multiplies the other parameter with the unsharp signal, and adds the two resulting signals to obtain the final image signal. One embodiment requires that the sum of the two parameters equal one for all image signal levels. In this case, the method is mathematically equivalent to the unsharp mask equation.
Shimazaki teaches that the two parameters are signal dependent with the signals representing image highlights resulting in the highest degree of sharpening. The two parameters are chosen such that the sharpening decreases as either the image signal or the unsharp signal decreases until the sharpening level is zero. At that point, the sharpening converts to blurring as the image signal or unsharp signal continue to decrease into the shadow region of the density range. Shimazaki's apparatus suffers from not accounting for different film speeds, film formats, and exposure levels.
Greensite U.S. Pat. No. 4,991,092 describes an imaging technique for enhancing the contrast of medical images which is dependent on measured noise. The noise is measured on a flat field part of the image and the results are used to modify the contrast enhancement. The technique is very restrictive because each image requires the presence of a suitable flat field area.
These examples have only discussed optimizing the sharpening level with respect to noise in the digital image. None of them have considered the effect of the magnification of the image when making the final print. Magnification is necessary to convert the image from a small format, such as 35 mm, on a color negative or reversal film to a large format, such as 4".times.6", which can be easily viewed. None of the prior art considers the effect of viewing condition for the final image. The proper optimization of sharpening must account for the noise, the magnification level, and the viewing condition of the final image.